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DETAILED ACTION 
Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-6, 10, 12-16, 21-24 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Daniell et al (6,678,421 ) in view of Yang et al (1 999 IEEE, A 
Semantic classification and composite indexing approach to robust image retrieval). 
Regarding method and computer readable medium claims 1,13 and 23 Daniell 
discloses, 

Obtaining an image and subdividing such image into multiple hierarchical layered 
blocks (note col. 5 lines 41-45 and 63-67, col. 6 lines 1-6, decomposing original image 
into multiple levels blocks), the blocks within a layer being at least partially coextensive 
(see fig. 2 and col. 6 lines 1-6, in multilevel decomposition, smaller block is a smaller 
part of the larger block, i.e. level 2 is coextensive to part of level 1); 

Determining a posterior estimate of class membership of hierarchical layered 
blocks (as described in specification page 28 lines 18-19, examiner interprets 
determining posterior estimates of class membership of hierarchical layered blocks as 
using Bayes rule based on multilevel sub bands (note col. 7 lines 7-28), the estimated 
being bases upon class likelihoods of the hierarchical layered blocks in the group (note 
col. 7 lines 7-28, Bayes rules is based upon the probability of significant and 
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insignificants of the level sub bands), such likelihood being condition on data extracted 
from hierarchical layered blocks in the group (note col. 3 lines 3-9, col. 6 lines 5-28 and 
col. 7 lines 1-20, probability being condition on predicted coefficients from sub band 
levels). 

Classifying a portion of such image based upon the posterior estimate of class 
membership conditioned on the data extracted from the group of hierarchical layered 
blocks local to such portion (note fig. 7 choose class z and col. 6 lines 7-27, 56-65 col. 7 
lines 49-65, classifying high frequency signal using Bayes rules of low frequency 
information (sub band levels)). 

Although Daniell discloses coefficient prediction scheme of sub bands based on 
content of the input, which is the source of the information. Daniell does not clearly 
disclose semantically classifying a portion of the image. Yang et al discloses 
semantically classifying an image (note page136, col. 1 lines 1-4, images similar 
semantic meanings wide variety of low-level features). Daniell and Wang are 
combinable because they disclose classifiers. At the time of the invention, it would have 
been obvious to a person of ordinary skills in the art to semantically classify a portion of 
an image in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 1 . 
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Regarding claim 2 Daniell discloses, 

Determining an estimated class likelihood of each block in the group of 
hierarchical layered blocks (note col. 7 lines 1-27 and col. 8 lines 6-8, lines cite p (m, z) 
a likelihood slope value generating an insignificant coefficient each base on coefficient 
of each band in the group of levels); 

Combining the estimated class likelihood of hierarchical layered blocks in the 
group into a posterior estimate of class membership (note col. 7 lines 23-27, a joint 
probability is formed by combining p (m, z) P (z) and p (m, nz) into Bayes rule). 

Although Daniell discloses coefficient prediction scheme of sub bands based on 
content of the input, which is the source of the information. Daniell does not clearly 
disclose semantically classifying. Yang et al discloses semantically classifying (note 
page136, col. 1 lines 1-4, images similar semantic meanings wide variety of low-level 
features). Daniell and Wang are combinable because they disclose classifiers. At the 
time of the invention, it would have been obvious to a person of ordinary skills in the art 
to semantically classify in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 2 



Regarding claim 3 Daniell discloses, 
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Repeating the classifying for each of multiple portions that substantially covers 
the entire image (note col. 9 lines 30-48, algorithm include classifying of sub bands over 
an image, in one level and applied to recursively to other levels). 

Categorizing the entire image based upon the result of such repeating classifying 
(note col. 9 lines 30-48, coefficient are classified over the image based on recursive 
algorithm). 

Regarding method claim 4 Daniell discloses, 

Wherein each successively lower layer of the multiple hierarchical layered blocks 
is comprised of one or more blocks which are smaller than and at least partially 
coextensive with one or more blocks in a layer immediately above (fig. 2, multilevel 
decomposition shows layered blocks comprised of one or more blocks which are 
smaller than at leas partially coextensive a layer immediately above). 

Regarding claim 5 

Although Daniell discloses, extracting features from blocks. Daniel does not 
clearly disclose extracting low-level features from blocks. Yang discloses extracting 
low-level features (note page 136 col. 1 lines 1-4 and page 137 col. 1 lines 19-24, 
images containing low-level feature vectors). Daniell and Wang are combinable 
because they disclose classifiers. At the time of the invention, it would have been 
obvious to a person of ordinary skills in the art to extract low-level features in the system 
of Daniell as evidenced by Yang et al. The suggestion/motivation for doing so would 
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have been improving image classification and bridging the gap between low-level 
features and high-level semantic meanings (note page 134, col. 2 lines 3-6). Therefore, 
it would have been obvious to combine Daniell with Yang to obtain the invention as 
specified in claim 5. 

Regarding method claim 6 

Although Daniell discloses, extracting features from blocks. Daniel does not 
clearly disclose extracting low-level features comprise color and texture. Yang 
discloses low-level features comprise color and texture (note page 136 paragraph 3 
lines 5-13 and page 137 col. 1 lines 1-4, feature background extraction low level where 
color is relevant and texture class texture is relevant). Daniell and Wang are 
combinable because they disclose classifiers. At the time of the invention, it would have 
been obvious to a person of ordinary skills in the art to extract low-level features 
comprising color and texture in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 6. 

Regarding claim 10 Daniell discloses, 

Wherein a classification that results from the classifying step is binary (note col. 
1 1 lines 38-40, shows results as binary strings). Daniell does not clearly disclose 
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semantically classifying. Yang et al discloses semantically classifying (note page136, 
col. 1 lines 1-4, images similar semantic meanings wide variety of low-level features). 
Daniell and Yang are combinable because they disclose classifiers. At the time of the 
invention, it would have been obvious to a person of ordinary skills in the art to 
semantically classifying in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 10. 

Regarding method claim 12 

Daniell discloses an image classification method. Daniell does not clearly 
disclose searching for images matching a given query in an image library containing 
images having portions thereof classified using the semantic image classification 
method. Yang et al discloses searching for images matching a given query in an image 
library containing images having portions thereof classified using the semantic image 
classification method (note page 136 paragraph 3 lines 1-4, page 137, paragraph 4 
lines 21-35 and page138 col. 1 lines 1-20, images are search matching a query seed 
from databases using semantic image classification). Daniell and Yang are combinable 
because they disclose classifiers. At the time of the invention, it would have been 
obvious to a person of ordinary skills in the art to search for images matching a given 
query in an image library containing images having portion thereof classified using 
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semantic image classification in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings and for 
performance of retrieval system multi-feature templates performs better than other 
templates (note page 134, col. 2 lines 3-6 and page 138 col. 2 lines 1-10). Therefore, it 
would have been obvious to combine Daniell with Yang to obtain the invention as 
specified in claim 12. 

Regarding system and computer readable claims 14 and 22, respectively Daniell 
discloses, 

Estimate class likelihoods for each block (note col. 7 lines 7-28, Bayes rules 
estimates the probability of each sub bands), a class being discriminating classification 
(class being significant and insignificant features of the sub bands) and a block being 
portion of the image (sub band is a portion of the image); 

Generates a posterior estimated of class membership based on combining 
estimated class likelihoods of hierarchical sets of blocks (note col. 7 lines 23-27, a joint 
probability is formed by combining p (m, z) P (z) and p (m, nz) into Bayes rule), a 
hierarchical set of blocks being hierarchical organized (note col. 5 lines 41-45 and 63- 
67, col. 6 lines 1-6, decomposing original image into hierarchical set of blocks (multiple 
levels blocks)) and associated blocks that are, at least partially coextensive (see fig. 2 
and col. 6 lines 1-6, in multilevel decomposition, smaller block is a smaller part of the 
larger block, i.e. level 2 is coextensive to part of level 1 ); 
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Determine and classify one of multiple discriminating classifications to localize 
portions of the image based upon the posterior estimate of class membership of blocks 
comprising such portions (note fig. 7 choose class z and col. 6 lines 7-27, 56-65 col. 7 
lines 49-65, classifying high frequency signal using Bayes rules of low frequency 
information (sub band levels)). 

Although Daniell discloses coefficient prediction scheme of sub bands based on 
content of the input, which is the source of the information. Daniell does not clearly 
disclose semantically classifying. Yang et al discloses semantically classifying (note 
page136, col. 1 lines 1-4, images similar semantic meanings wide variety of low-level 
features). Daniell and Wang are combinable because they disclose classifiers. At the 
time of the invention, it would have been obvious to a person of ordinary skills in the art 
to semantically classify in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 14 

Although Daniell discloses, extracting features from blocks. Daniel does 
not clearly disclose extracting low-level features from blocks. Yang discloses extracting 
low-level features (note page 136 col. 1 lines 1-4 and page 137 col. 1 lines 19-24, 
images containing low-level feature vectors). Daniell and Wang are combinable 
because they disclose classifiers. At the time of the invention, it would have been 
obvious to a person of ordinary skills in the art to extract low-level features in the system 
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of Daniell as evidenced by Yang et al. The suggestion/motivation for doing so would 
have been improving image classification and bridging the gap between low-level 
features and high-level semantic meanings (note page 134, col. 2 lines 3-6). Therefore, 
it would have been obvious to combine Daniell with Yang to obtain the invention as 
specified in claim 14. 

Regarding system claim 15 Daniell discloses, 

Wherein each successively lower layer of the multiple hierarchical layered blocks 
is comprised of one or more blocks which are smaller than and at least partially 
coextensive with one or more blocks in a layer immediately above (fig. 2, multilevel 
decomposition shows layered blocks comprised of one or more blocks which are 
smaller than at leas partially coextensive a layer immediately above). 

Regarding system claim 1 6 

Although Daniell discloses, extracting features from blocks. Daniel does not 
clearly disclose extracting low-level features comprise color and texture. Yang 
discloses low-level features comprise color and texture (note page 136 paragraph 3 
lines 5-13 and page 137 col. 1 lines 1-4, feature background extraction low level where 
color is relevant and texture class texture is relevant). Daniell and Wang are 
combinable because they disclose classifiers. At the time of the invention, it would have 
been obvious to a person of ordinary skills in the art to extract low-level features 
comprising color and texture in the system of Daniell as evidenced by Yang et al. The 
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suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 16 

Regarding system claim 21 

Daniell discloses an image classification method. Daniell does not clearly 
disclose searching for images matching a given query in an image library containing 
images having portions thereof classified using the semantic image classification 
method. Yang et al discloses searching for images matching a given query in an image 
library containing images having portions thereof classified using the semantic image 
classification method (note page 136 paragraph 3 lines 1-4, page 137, paragraph 4 
lines 21-35 and page138 col. 1 lines 1-20, images are search matching a query seed 
from databases using semantic image classification). Daniell and Yang are combinable 
because they disclose classifiers. At the time of the invention, it would have been 
obvious to a person of ordinary skills in the art to search for images matching a given 
query in an image library containing images having portion thereof classified using 
semantic image classification in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings and for 
performance of retrieval system multi-feature templates performs better than other 
templates (note page 134, col. 2 lines 3-6 and page 138 col. 2 lines 1-10). Therefore, it 
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would have been obvious to combine Daniell with Yang to obtain the invention as 
specified in claim 21. 

Regarding computer readable claim 24 Daniell discloses, 

Obtaining an image and subdividing such image into multiple hierarchical layered 
blocks (note col. 5 lines 41-45 and 63-67, col. 6 lines 1-6, decomposing original image 
into multiple levels blocks), the blocks within a layer being at least partially coextensive 
(see fig. 2 and col. 6 lines 1-6, in multilevel decomposition, smaller block is a smaller 
part of the larger block, i.e. level 2 is coextensive to part of level 1); 

Determining a posterior estimated of class membership by combining estimated 
likelihoods of hierarchical layered of blocks (note col. 7 lines 23-27, a joint probability is 
formed by combining p (m, z) P (z) and p (m, nz) into Bayes rule of each level), 

Classifying a portion of such image based upon the posterior estimate of class 
membership conditioned on the data extracted from the group of hierarchical layered 
blocks local to such portion (note fig. 7 choose class z and col. 6 lines 7-27, 56-65 col. 7 
lines 49-65, classifying high frequency signal using Bayes rules of low frequency 
information (sub band levels)). 

Although Daniell discloses coefficient prediction scheme of sub bands based on 
content of the input, which is the source of the information. Daniell does not clearly 
disclose semantically classifying a portion of the image. Yang et al discloses 
semantically classifying an image (note page136, col. 1 lines 1-4, images similar 
semantic meanings wide variety of low-level features). Daniell and Wang are 
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combinable because they disclose classifiers. At the time of the invention, it would have 
been obvious to a person of ordinary skills in the art to semantically classify a portion of 
an image in the system of Daniell as evidenced by Yang et al. The 
suggestion/motivation for doing so would have been improving image classification and 
bridging the gap between low-level features and high-level semantic meanings (note 
page 134, col. 2 lines 3-6). Therefore, it would have been obvious to combine Daniell 
with Yang to obtain the invention as specified in claim 24. 

3. Claims 7 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Daniell et al. and Yang et al. in further view of Ohta et al ("Color information for region 
segmentation", Computer Graphics and Image Processing, 13:222-241, 1980). 
Regarding method and system claims 7 and 17, respectively, 

Although Daniell modified discloses extracting color low-level features. Daniell 
modified does not clearly disclose extracting color low-level features by Ohta 
decomposition. Ohta discloses extracting color low-level features by Ohta 
decomposition (note fig. 1, page 225, and fig. 4 page 230 and 231, describes extracting 
of color features by transformation process for segmenting images). Daniel modified 
and Ohta are combinable because they disclose segmenting of color features. At the 
time of the invention, it would have been obvious to a person of ordinary skill in the art 
to disclose extracting color low-level features by Ohta decomposition in the system of 
Daniell modified as evidenced by Ohta et al. The suggestion/motivation for doing so 
would have been extracting non- degraded color features (page 230 last paragraph). 
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Therefore, it would have been obvious to combine Daniell modified with Ohta et al to 
obtain the invention specified in claims 7 and 17. 

4. Claims 8 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Daniell et al. and Yang et al. in further view of Hatipoglu et al ("Texture classification 
using dual-tree complex wavelet transform", IEEE, 1999, 344-347). 
Regarding method and system claims 8 and 18, respectively, 

Although Daniell modified discloses extracting texture low-level features. Daniell 
modified does not clearly disclose extracting texture low-level features by complex 
wavelet transform. Hatipoglu et al discloses extracting texture low-level features by 
complex wavelet transformation (note page 345-346 paragraph 3, describes extracting 
texture features by dual tree complex wavelet transform). Daniel modified and Hatipoglu 
are combinable because they disclose segmenting of texture features. At the time of 
the invention, it would have been obvious to a person of ordinary skill in the art to 
disclose extracting texture low-level features by Complex wavelet transform in the 
system of Daniell modified as evidenced by Hatipoglu et al. The suggestion/motivation 
for doing so would have been improved image restoration and enhancement and 
obtaining directional selectivity of features and separable filtering (note page 344 col. 1 
lines 22-23 and page 345 col. 1 lines 26-27). Therefore, it would have been obvious to 
combine Daniell modified with Hatipoglu et al to obtain the invention specified in claims 
8 and 18. 
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5. Claims 9 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Daniell et al. and Yang et al. in further view of Lu (6,317,517). 

Regarding method and system claims 9 and 19, respectively, 

Although Daniell modified discloses class likelihoods in dimensional space. 
Daniell modified does not clearly disclose class likelihoods estimated in one- 
dimensional space. Lu discloses reducing dimensions of data (note fig. 1 block 12 and 
col. 2 lines 8-25, reducing dimensions of data as described in specification page 29 
lines 2-6). Daniel modified and Lu are combinable because they disclose class 
likelihood using Bayes rule. At the time of the invention, it would have been obvious to 
a person of ordinary skill in the art to disclose class likelihood estimated in one- 
dimensional space in the system of Daniell modified as evidenced by Lu. The 
suggestion/motivation for doing so would have been improved statistical pattern 
recognition (note col. 2 lines 1-5). Therefore, it would have been obvious to combine 
Daniell modified with Lu to obtain the invention specified in claims 9 and 19. 

6. Claims 1 1 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Daniell et al. and Yang et al. in further view of Syeda-Mahmood (6,594,383). 
Regarding method and system claims 1 1 and 20, respectively, 

Although Daniell modified discloses classification that results from semantically 
classifying step is binary. Daniell modified does not clearly disclose a group consisting 
of sky or not-sky; grass or not-grass; natural or manmade; inside or outside; hair or not- 
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hair; face or not-face. Syeda-Mahmood discloses a group consisting of sky or not-sky; 
grass or not-grass; natural or manmade; inside or outside; hair or not-hair; face or not- 
face (note col. 5 lines 24-45, lines cite classifying surface class sky and grass, natural 
and manmade lightings, outdoor and indoor lightings). Daniel modified and Syeda- 
Mahmood are combinable because they disclose semantically classification. At the 
time of the invention, it would have been obvious to a person of ordinary skill in the art 
to disclose classifying a group consisting of sky or not-sky; grass or not-grass; natural or 
manmade; inside or outside; hair or not-hair; face or not-face in the system of Daniell 
modified as evidenced by Syeda-Mahmood. The suggestion/motivation for doing so 
would have been increase statistical variations for more accurate surface recognition 
(note col. 5 lines 20-21 , 37-40 and 46-48). Therefore, it would have been obvious to 
combine Daniell modified with Syeda-Mahmood to obtain the invention specified in 
claims 11 and 20. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gregory M. Desire whose telephone number is (571) 
272-7449. The examiner can normally be reached on M-F (6:30-3:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571) 272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Gregory M. Desire 

Examiner 

Art Unit 2625 



G.D. 




September 29, 2005 



